Papers with automatic shortcut detection
Finding Dataset Shortcuts with Grammar Induction (2022.emnlp-main)
Copied to clipboard
| Challenge: | Prior work on shortcut detection focused on enumerating features like unigrams or bigrams . prior work relied on post-hoc models that reveal qualitative patterns without a clear statistical interpretation . |
| Approach: | They propose to use probabilistic grammars to characterize and discover shortcuts in NLP datasets using context-free grammars and synchronous context- free grammars. |
| Outcome: | The proposed grammars reveal interesting shortcut features in a number of datasets, including simple and high-level features, and automatically identify groups of test examples on which conventional classifiers fail. |